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Introduction (iof3> 


Author/Presenter: John A. Latimer 

• Senior System Reliability Engineer 

• Over 33 Years Experience in Systems Engineering 
and Risk Management 

■ Concept Formulation 

■ Development 

■ Test 

■ Production 

• Employed at Science Applications International 
Corporation (SAIC) for Over 22 Years 

• Currently Working the National Aeronautics and 
Space Administration (NASA) Expendable Launch 
Vehicle (ELV) Contract at Kennedy Space Center 
(KSC) 



Introduction (2of3> 


Presentation Material : 

• Residual Risk Evaluation Technique (RRET) 

■ RRET Was Developed by KSC’s Safety and Mission 
Assurance (SMA) Launch Services Division to Provide 
System Reliability Input to the Decision Makers for ELV's 
Readiness Reviews and Other Major Life Cycle Milestones 

• RRET Determines the Impact of Residual Risks on the 
System Baseline Reliability Throughout the ELV's Life 
Cycle Mission 

■ RRET Met the Approval of the Office of Safety and Mission 
Assurance (OSMA) at NASA Headquarters 
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Introduction (3of3> 


ELV’s Risk Management: 

• A Continuous Risk Management Plan (RMP) Is Being 
Implemented for Each ELV Mission 

■ LSP-PLIN-353-01,” Launch Services Program Risk 
Management Plan” 

o The RMP Is Based on NASA Requirements and 
Guidelines 

- NPR 8000.4, “Risk Management Procedural 
Requirements” 

- NPD 8700. 1C, “NASA Policy For Safety And Mission 
Success” 

o The RMP Includes a Controlled, Logical, Management 
Procedure for Identifying, Assessing, and Reporting 
Potential Technical Risks 

o SMA Performs Reliability Analysis as One of Many 
Independent Mission Assurance Tasks to Maximize 
Mission Success for Each ELV Mission 


■ _ 1-! 


^ a f ai ■ i 


T 


■ _ » ■ ■ 


Applied Reliability Symposium, North America 2009 


l RRET Overview 


RRET: 

• A Simplistic, Cost Effective Residual Risk Evaluation 
Technique 

■ It Provides Quantifiable Insight Into the Severity of the 
Residual Risks Impact on a System Baseline Reliability 

• The System Reliability Impact Indicator Provides a Quantitative 
Measure of the Reduction in the System Baseline Reliability Due 

to the Identified Residual Risks 

• Proven Methodology 

■ Risk Management 

• NASA: Risk Management Procedural Requirements 

■ Fault Tree Analysis (Probability of Failure - P F ) 

• NASA: Fault T ree Handbook with Aerospace Application 

■ Reliability Prediction (Probability Of Success - R) 

• NASA: System Engineering Toolbox for Design-Oriented 
Engineers 
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RRET Implementation Process (iof6> 


The RRET Implementation Process Involves 5 Steps: 

• Step 1 : Generation of System Baseline Reliability 
(R SB r) Prediction [3] 

■ RRET Uses Standard Industry Procedure (MIL HDBK 338) 

o Define the Configuration for Which the Prediction Is Applicable 
o Define the Service Use (Life Cycle) 
o Define and Generate the Item Reliability Block Diagrams 
o Define the Mathematical Models for Computing Item Reliability 
o Define the Parts of the Item 

o Define the Environmental Profile and Expected Conditions 
o Define the Stress Conditions 
o Define the Failure Distribution 
o Define the Failure Rates 
o Compute the Item Reliability 
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RRET Implementation Process < 2 ore) 


• Step 2: Transformation of the RBD Model to a Fault 
Tree Model [8] 

■ Fault Tree Generation 
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RRET Implementation Process (3of6> 

• Step 2: Transformation of the RBD Model to a Fault 

T ree Model (Continued) 

■ Calculate the Corresponding Failure Probabilities 
o Perform Analyses Per Fault Tree Software Package 
- System Failure Probability (P F = 1 - R) 
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RRET Implementation Process <4of6) 

• Step 3: Determination of Failure Probabilities for 
Residual Risks and Uncertainty Events [3] 

■ Residual Risks Sources 

o Manufacturer’s 

o MIL-Standards 

o Historical Data (Similarity Theory) 
o Simulation Data 
o T est Data 
o Industry Standards 
o Delphi Technique 

■ Mitigation Plan Uncertainty Events Source 

o Delphi Technique 
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RRET Implementation Process (5of6> 


• Step 4: Generation of Residual Risks Fault Tree [2] 

■ Construct Fault Tree Using Software Package 

o Integrate Residual Risks and Mitigation Plan Uncertainty Events Into 
the Baseline Fault Tree 

- Residual Risks and Designated Mitigation Plan Uncertainty 
Events Are Propagated Through an AND Gate 

o Determine System Failure Probability (Psrrf) 
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RRET Implementation Process (6 of 6) 


• Step 5: Determination of Residual Risk Indicator 

■ Calculate the System Residual Risk Reliability (Rsrrr) 
Parameter 

o Rsrrr = 1 - Psrrf 

■ Derive the Residual Risk Indicator 

o Indicator = Rsbr - Rsrrr 
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Bf Step 1 : Generation of System Baseline 
Reliability Prediction, R qRR (1 of 2) 
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8J Step 1: Generation of System Baseline 
Reliability Prediction, R qRR (2 of 2) 


Reliability Block Diagram Model: 


System Baseline Reliability (R sbr ) = 

Rsi* R S2 * R S3 = 0 -982975 

Rsi 

Rs2 

Rs3 


Subsystem 1 (SI) 

Subsystem 2 (S2) 

Subsystem 3 (S3) 

Rsbr 

0.987615 

0.999500 

0.995800 



System Level Baseline Reliability (R sbr ) 


Rsi “ Rai* Ra2* Ra3 _ 0.98761 5 


Rai 

Ra2 

Ra3 


Assembly 1 (A1 ) 

Assembly 2 (A2) 

Assembly 3 (A3) 

Rsi 

0.997137 

0.995754 

0.994674 



Subsystem Level Reliability (R S1 ) 
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Step 2: Transformation of the RBD 
Model to a Fault Tree Model 


Fault Tree Diagram 


System Failure Probability (Pf) = 0.017025 


System Failure 


♦ 


Baseline Model 


S 




Subsystem 3 

Subsystem 1 


Subsystem 2 

Failure 

Failure 


Failure 

S3 

SI 


S2 

♦ 4 

* 


i 


1 

Assembly 1 

Assembly 2 


Assembly 3 

Failure 

Failure 


Failure 

Assembly-1 

Assembly-2 


Assembly-3 
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Step 3: Determination of Failure Probabilities 
for Residual Risks and Uncertainty Events 


Residual 

Risk 

Failure 

Probability 

Data Source 

Affected 

Subassembly 

Risk Item 1 

s 1.760E-03 

Historical Data and 
Manufacturer 

A1SA1 

Risk Item 2 

* 2.000E-04 

Manufacturer and Test 
Data 

A1SA1 

Risk Item 3 

* 1 .937387E-03 

Historical Data and Test 
Data 

A1SA3 

Risk Item 4 

* 4.271 0E-03 

Mil-Standard and 
Historical Data 

A2SA2 

Risk Item 5 

* 1.990E-05 

Test Data 

A2SA4 


*To Simplify the Example, a Failure Probability Of .1 Was Used for the Mitigation 
Uncertainty Events... Delphi Technique 



Step 4: Generation of Residual Risks 


Fault Tree < 

1 of 6) 


Fault Tree Diagram (continued) 



System Failure Probability (Psrrf) = 0.017829 
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Sf Step 4 : Generation of a Residual 
Risk Fault Tree <2of6i 


Fault Tree Diagram (continued) 
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Step 4: Generation of a Residual 
Risk Fault Tree oo fe> 


Fault Tree Diagram (continued) 
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Step 4: Generation of a Residual 


Risk Fault Tree uae) 


Fault Tree Diagram (continued) 
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Step 4: Generation of a Residual 
Risk Fault Tree isotei 


Fault Tree Diagram (continued) 
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Step 4: Generation of a Residual 
Risk Fault Tree teofei 


Fault Tree Diagram (continued) 
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if Step 5: Determination of Residual 
Risk Indicator 

• Calculate the System Residual Risk Reliability 
(Rsrrr) Parameter 

" PsRRR = 1 " P SRRF = 0.9821 71 

• Derive the Residual Risk Indicator 

■ Indicator = Rsbr " Psrrr = 

0.982975 - 0.982171 = 0.000804 
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Summary 


• KSC’s S&MA Launch Services Division Developed a Residual 
Risk Evaluation Technique for Reliability Insight 

■ It Is a Simplistic, Cost Effective Technique That Provides Decision Makers 
a Quantifiable Insight into the Severity of the Cumulative Residual Risks 
Impact Associated with any System. 

■ The Quantifiable Insight Is Determined by Using the Proven Methodology 

o Risk Management 
o Reliability Prediction 
o Fault Tree Analysis 

■ RRET Calculates the Reduction In System Baseline Reliability Due to 
Identified Residual Risks. 

• A Simple System Was Provided As an Example to Show RRET’s 
Application 

• RRET Can Be Adapted to a Wide Variety of Complex Systems, 
Processes, and Facilities 
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Where to Get More Information 
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John A. Latimer 


• John A. Latimer is currently a Senior System Lead Reliability 
Engineer at Science Applications International Corporation 
(SAIC) working on the NASA Expendable Launch Vehicle 
Contract. He has over 33 years experience in Systems 
Engineering and Risk Management with expertise in the 
Specialty System Engineering areas of Reliability, 
Maintainability, and Availability (RMA). His experience 
encompasses concept formulation, development, integration, 
test, production, and fielding of military and commercial 
systems. 

• Education/Contact Information: 

■ Mr. Latimer received his BS and MS degrees in Electrical Engineering 
from Tennessee State University and Stanford University, respectfully 

■ E-mail Address: John.A.Latimer@nasa.gov or Latimeri@saic.com 

■ Address: Mailcode Analex-3, Kennedy Space Center, FL 32899 

■ Phone Number: 321-867-8719 
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Questions 


Thank You for Your Attention 

Do You Have Any Questions? 
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Attachmentl :Definitions 


(1 of 2) 


• Assembly is an item composed of any number of parts or subassemblies, 
joined together to perform a specific function, which can be disassembled 
without destruction. 

• Assessment is an evaluation or appraisal of the state of a system, program/ 
project or a portion of a program/project. 

• Delphi Technique is an iterative process that results in a consensus by a 
group of experts. 

• Fault Tree Analysis is a deductive system reliability tool which provides both 
qualitative and quantitative measures of the probability of failure. It estimates 
the probability that a top level event will occur, systematically identifies all 
possible causes leading to the top event, and documents the analytic process 
to provide a baseline for future studies of alternative designs. 

• Human Error Risk Assessment is a process that identifies risks to designs, 
equipment, procedures, and tasks as a result of human error. 

• Mission Reliability is the measure of the ability of an item to perform its 
required function for the duration of a specified mission profile. Mission 
reliability defines the probability that the system will not fail to complete the 
mission, considering all possible redundant modes of operation. 
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Attachment! :Definitions (2of2> 


• Reliability Prediction is a forecast of the reliability of a system or system 
element, postulated on analysis, past experience, and tests. 

• Residue Risk is the risk that remain after risk management options have been 
identified and the required mitigation plans implemented properly. 

• Risk is a combination of the likelihood of an undesirable event occurring and 
the severity of the consequences of the occurrence. 

• Risk Assessment, Quantitative is the process of assigning proportional 
numerical quantities to both the likelihood and the adverse consequences of 
risk items. 

• Risk Management is an organized means of controlling the risk on a program. 

• Risk Mitigation is the process of reducing either the likelihood or the severity 
of a risk because the benefits from assuming the risk do not outweigh the 
perceived risk. 

• Subsystem is a grouping of items satisfying a logical group of functions within 
a system. 

• System is an integrated aggregation of end items, interfaces, and support 
functions designed to fulfill a specific mission requirement. A system may 
include equipment, trained personnel, facilities, data and procedures, and 
software. For program/project purposes, a system is typically defined as the 
highest level of hardware organization composed of multiple subsystems. 
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Attachment 2:Acronyms (ion) 


• ELV - Expendable Launch Vehicle 

• FTA - Fault T ree Analysis 

• FT -Fault Tree 

• KSC - Kennedy Space Center 

• OSMA - Office of Safety and Mission Assurance 

• P F - Probability of Failure 

• P s - Probability of Success 

• RBD - Reliability Block Diagram 

• NASA - National Aeronautics and Space Administration 

• NPD - NASA Policy Directive 

• NPG - NASA Procedures and Guidelines 

• Rsbr - System Baseline Reliability 

• Rsrrr ‘ System Residual Risk Reliability 

• RMP - Risk Management Program 

• RRET - Residual Risk Evaluation Technique 

• SMA - Safety And Mission Assurance 
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